Object change detection and measurement using digital fingerprints

ABSTRACT

The present disclosure teaches a method of utilizing image “match points” to measure and detect changes in a physical object. In some cases “degradation” or “wear and tear” of the physical object is assessed, while in other applications this disclosure is applicable to measuring intentional changes, such as changes made by additive or subtractive manufacturing processes, which may, for example, involve adding a layer or removing a layer by machining. A system may include a scanner, and a digital fingerprinting process, coupled to an object change computer server. The server is coupled to a datastore that stores class digital fingerprints, selected object digital fingerprints collected over time, match measurements, and deterioration metrics.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a non-provisional of and claims priority, pursuantto 35 U.S.C. § 119(e), to U.S. Application No. 62/802,177, filed Feb. 6,2019 (ref. 0680-P), hereby incorporated by reference as though fully setforth.

COPYRIGHT NOTICE

© Alitheon, Inc. 2019-2020. A portion of the disclosure of this patentdocument contains material which is subject to copyright protection. Thecopyright owner has no objection to the facsimile reproduction by anyoneof the patent document or the patent disclosure, as it appears in thePatent and Trademark Office patent file or records, if and when they aremade public, but otherwise reserves all copyright rights whatsoever. 37CFR § 1.71(d).

FIELD OF THE DISCLOSURE

This application pertains to methods, systems, and software fordetecting and measuring changes over time to physical objects, throughthe use of digital fingerprints.

BACKGROUND OF THE DISCLOSURE

It is often necessary or desirable to determine whether or not aphysical object has changed from a prior state and, in certain cases, todetermine how much a physical object has changed from a prior state. Theneed remains to reliably identify objects, detect and measure changes inthe object over time, and from those changes assess authenticity,provenance, quality, condition, and/or degradation over time.

SUMMARY OF THE DISCLOSURE

The following is a summary of the present disclosure to provide a basicunderstanding of some features and context. This summary is not intendedto identify key or critical elements of the disclosure or to delineatethe scope of the disclosure. Its sole purpose is to present someconcepts of the present disclosure in simplified form as a prelude to amore detailed description that is presented later.

In one embodiment, a system comprises a combination of digitalfingerprint techniques, processes, programs, and hardware to enable amethod of utilizing image “match points” to measure and detect changesin a physical object. In some cases “degradation” or “wear and tear” ofthe physical object is assessed, while in other applications thisdisclosure is applicable to measuring intentional changes, such aschanges made by additive or subtractive manufacturing processes, whichmay, for example, involve adding a layer or removing a layer bymachining. A system may include a scanner, and a digital fingerprintingprocess, coupled to an object change computer server. The object changecomputer server may be coupled to a datastore that stores class digitalfingerprints, selected object digital fingerprints collected over time,match measurements, and deterioration metrics.

Additional aspects and advantages of the present disclosure will beapparent from the Detailed Description, which proceeds with reference tothe accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

To enable the reader to realize one or more of the above-recited andother advantages and features of the present disclosure, a moreparticular description follows by reference to specific embodimentsthereof which are illustrated in the appended drawings. Understandingthat these drawings depict only typical embodiments of the disclosureand are not therefore to be considered limiting of its scope, thepresent disclosure will be described and explained with additionalspecificity and detail through the use of the accompanying drawings inwhich:

FIG. 1 is a simplified block diagram of one example of a system todetect and measure changes in physical objects using digitalfingerprints.

FIG. 2 is an image of a portion of a dollar bill with digitalfingerprinting points of interest superimposed as circles on the image.

FIG. 3 is an image of a portion of a turbine blade with digitalfingerprinting points of interest superimposed as circles on the image.

FIG. 4 is a simplified flow diagram illustrating a first process todetect and measure degradation in physical objects using digitalfingerprints.

FIG. 5 is a simplified flow diagram of a second process to detect andmeasure degradation in physical objects using digital fingerprints.

DETAILED DESCRIPTION OF ONE OR MORE EMBODIMENTS

It is often necessary or desirable to determine whether or not aphysical object has changed from a prior state and, in certain cases, todetermine how much a physical object has changed from a prior state. Thepresent disclosure teaches a method of utilizing image “match points” tomeasure and detect changes in a physical object. While some aspects ofthe results presented herein may be achieved by examining changes inimages that are not changes in match points, existing image-basedapproaches are significantly less general (for additive as well assubtractive processes, for example), and thus more limited, than themethods taught in this disclosure.

In this disclosure, frequent reference is made to “degradation” or “wearand tear” but the teachings of this disclosure are equally applicable tomeasuring intentional changes, such as changes made by additivemanufacturing processes, which may, for example, involve adding a layeror removing a layer by machining, and any other processes that result inchanges to the state of an object.

One aspect of this disclosure is the use of measured changes in adigital fingerprint of an object as a measure of its actual change fromits initial conditions, or any prior condition. The techniques areprimarily described in relation to measuring wear and tear anddetermining that an object should be replaced, but the techniquesdescribed here also have applicability in measuring the effects of otherchanges such as the effectiveness of anodizing a surface or an additivemanufacturing process in, for example, adding a layer or a feature to anobject under construction—or to any occurrence that results in a changein or to the material substance of a physical object. The methods taughtin the present disclosure may also be applied to the grading of objects,such as of a coin, based on a degradation of the object from a standard.One application of the taught method is determining when a currency notehas degraded to a such degree that it needs to be recycled.

As will be shown, the measured degradation may be from a digitalfingerprint created at an earlier stage of the object itself or from astandard representative of the kind (class) of object being measured, orby other means. This standard may be, for example, a particular object,a physical or digital model, or it may be an aggregate of many objectsof the same class. The concept of measured degradation will be furtherelaborated below.

During the creation and/or use of various physical goods, it is oftennecessary or desirable to determine that a physical good or object haschanged from some prior state—and to determine how much it has changedor in what manner it has changed. This disclosure teaches a method forusing digital fingerprints to determine and measure various objectchanges in a wide range of physical objects and under a range ofconditions. Examples will be given where the change is one ofwear-and-tear, corrosion, ablation, or other essentially subtractiveprocesses—where the surface elements are gradually removed. Equallyimportant are changes to the surface that occur through additiveprocesses, such as a step in an additive manufacturing process, or aperson's addition of makeup to his or her face. The additive conditions,the subtractive conditions, and conditions where both addition andsubtraction take place at the same time or in succession (e.g. abrasion,corrosion, and patination) are in view of this disclosure.

Wear and tear generally refers to changes in the surface or near-surfaceconditions that may be seen through electromagnetic interaction. Notethat the use of any part of the electromagnetic spectrum is in view ofthis disclosure, although most of the examples will involve visiblelight. No particular method of extracting digital fingerprints isrequired for this disclosure. However, the methods chosen will need tobe ones that extract the digital fingerprints from the surface ornear-surface features of the material substance of a physical object. Inthe present disclosure, the term “surface” will mean surface ornear-surface.

Wear and tear as defined in this disclosure consists of progressivechanges in the surface characteristics of a physical object. The changesmay be of many kinds including abrasion, creasing, soiling, aging,oxidizing, physical and/or chemical weathering, or any other progressivechange, including intentional changes such as plating. It is not to beimplied that changes to an object are necessarily negative or areindications of damage. Some of the embodiments discussed below involvean object “improving” (e.g. bringing the object closer to completion ofthe manufacturing process). Since digital fingerprints of the typedescribed are derived from surface characteristics of an object, changesin the surface characteristics may be quantified by measuring changes indigital fingerprints of the surface.

In this disclosure, frequent reference will be made to matching adigital fingerprint created at one point in time with a digitalfingerprint created at a different point in time and using the degree ofmatch as a component in the system. The matching may be done in variousways. As an example, an embodiment may carry out the matching of aplurality of digital fingerprints by calculating a simple thresholdfeature vector distance for each point of interest. Those closer thanthe threshold could be deemed to be a “match”. Other known types ofcalculations may be employed. See the description of DigitalFingerprinting, below, for more detail.

The technology disclosed herein may differentiate between several typesof changes in physical objects (based on the way the points of interestchange) and provide a measurement not only of overall change but ofdifferent types or categories of changes. Some examples, mentionedelsewhere in this disclosure, include distinguishing and measuringadditive and subtractive changes separately. Those examples are givenbut any kind of change that can be isolated based on digitalfingerprinting is in view of the taught system.

While this disclosure is not limited to the use of visible light, mostof the examples given use visible light. In most instances, the taughttechnology enables the use of visible light with no special equipment,which means that the hardware required to run the system may in manycases be consumer-level electronics (e.g. a smart phone). The ability touse relatively inexpensive equipment provides a considerable benefitover current approaches, many of which require specialized and oftenexpensive equipment.

In view in the disclosure is the ability of multiple users at differenttimes and places to contribute to a conceptual map of where (or when,how, why, etc.) changes to an object occur, which may enable a betterunderstanding of the history of the object and for that history to berecorded. For example, the system of FIG. 1 illustrates use of a remotesystem and smartphone (172) camera to capture image data of a physicalobject at virtually any time and location. Also, the taught system mayenable the detection of places where, for example, excessive wear isoccurring, thereby enabling possible fixes. Thus, the teachings of thisdisclosure may apply throughout a manufacturing process but also throughany process (such as a distribution network) wherein the object ishandled and may undergo intentional or unintentional change.

FIG. 1 is a simplified block diagram of one example of a system todetect and measure changes in physical objects using digitalfingerprints. In this example, a set of physical objects 100 of the sameclass are collected. The objects are presented one at a time into thefield of view of a suitable imager (scanner, camera, etc.) 102 toacquire image data. The image data is input to a digital fingerprintingprocess 104 to form a digital fingerprint for each object. The digitalfingerprinting process 104 may be integrated into the imager, stored inthe server 110, etc. It can be a remote process. The resulting set ofdigital fingerprints are provided via path 112 to an object changeserver 110. Any suitable computer server can be provisioned to functionas an object change server. It may be local or provisioned “in thecloud.” In this example, the server 110 is coupled by its communicationscomponent 150 to a network 160 which may be LAN, WAN, internet, etc.Almost any digital networking hardware and communication protocols canbe used.

A remote induction process 162 may be provisioned and coupled to theserver 110 via the network 160. This enables inducting, i.e., capturingimage data of an object from a remote location, and adding thecorresponding digital fingerprint of the remote object, to the server110. The server 110 may store digital fingerprint in a coupled datastore116, which again may be local or provisioned in the cloud. The objectchange server 110 can store data in the datastore 116 including digitalfingerprint data, to accomplish the functions described herein. Forexample, the datastore can maintain class digital fingerprints,individual (selected object) digital fingerprints, including multipleversions over time; reference object or digital fingerprints or digitalmodels of an object.

The matching process can be carried out, for example, by an analysiscomponent 144. It may use a query manager 142 to access various recordsas needed in the datastore 116. Results such as match measurements,deterioration metrics, and object histories and be stored and update inthe datastore. In one scenario, a remote user 170 may capture an objectimage using a smartphone 172. The image data can be used to generate acorresponding digital fingerprint, locally on the phone, or in the usersystem 170, or a remote induction component 162, or at the server 110.The object digital fingerprint can be stored in the datastore. It can beused to query the datastore to find a matching record from an earlierinduction. It can be used to compare to that earlier record, todetermine changes, and optionally to update the object history.

Objects with Substantially Similar Background Features

Two categories of objects are considered: First, those that are membersof a group of similar objects where a substantial portion of the digitalfingerprint of an individual object is shared by the similar objects;and second, those objects where there is no such sharing. Whether or notan object shares a “substantial portion” of the digital fingerprint withanother may be measured by a threshold value, may be defined by a modelor a template, or determined by some other means of measure. Shared orpotentially-shared portions of digital fingerprints are, for thepurposes of this disclosure, called “background features,” although thename is not critical or to be taken literally. In general (though notexclusively) the background features make different members of the grouplook similar even at fairly detailed levels of examination. A “fairlydetailed level of examination” of a physical object may utilize on theorder of 5-10× magnification, although these values are not critical. Anexample of a set of objects that has significant background features arecurrency bills. In fact, background features are commonlyintentionally-added characteristics in paper currency. For example, inthe case of a dollar bill, its background features are consistent enoughto distinguish a one-dollar bill from a twenty-dollar bill and wouldinclude such components as Washington's face and the numerical “1” atthe four corners of the bill.

FIG. 2 is an image of a portion of a dollar bill with digitalfingerprinting points of interest identified by superimposed circles onthe image. In this example, the points of interest are almost entirelylocated around “background” features. In one embodiment, the size of anindividual superimposed circle reflects the size of the point ofinterest. That is, the size of the circle is proportional to the scaleat which the corresponding point of interest was found. The scale of apoint of interest is the size of the radius around a point at which theLaplacian function (used to locate the point of interest) is strongestas one varies the size of the region.

In more detail, in one embodiment, the Laplacian is used to calculatethe curvature at each point on the surface. To do this, the Laplacianrequires two things: it needs to know the point at its center and itneeds to know how far out to look when calculating it. The latter is the“scale” of the Laplacian. One may pass a Laplacian of a given scale overthe surface and choose as the location of the points of interest thelocal (absolute value of the) maximum of the Laplacian. This is done forvarious scales. At a given point, as the Laplacian scale goes from verysmall to very large (over a pre-specified range of scales) there is somescale where its absolute value is largest. That value of the scale isthen chosen. The point of interest circle indicates that scale.

The examples and descriptions provided here are not meant to be limitingin any way but rather to provide a clear visual of the kind of objectbeing referred to. The digital fingerprints of currency bills often havepoints of interest whose locations are shared among many such bills andwhose characterizing feature vectors are relatively similar as well. Thereason for the similarity is that the object has features which areintentional features common to the group of objects and the digitalfingerprinting process is finding those common features. In general,such points of interest are close physically (meaning they are locatedat the same point on each dollar bill) but vary in characterization(since things like paper fibers and ink bleeds cause the feature vectorsof extracted digital features to vary significantly).

Objects that do not have Substantially Similar Background Features

The second category is objects that do not have significant sharedbackground features. Machined part surfaces commonly fall into thiscategory. To the naked eye, machined parts may appear to be extremelysimilar—for example, different brake pads of the same lot—but whenviewed at a resolution that on a dollar bill would show, for example,the background features of George Washington's face, such differentparts of the same kind show little or no common features. One reason isthat, for example, machined part surfaces may consist primarily ofcasting or machining marks which show little or no commonality acrossmembers of the type of object.

FIG. 3 is an image of a portion of a turbine blade with digitalfingerprinting points of interest superimposed as circles on the image.The image shows a section of a turbine blade. With very few exceptions,all the points marked on the photo are characteristics of the surface ofthe individual blade and not of a class of turbine blades. Very fewexceptions may be seen along the dark line across the image. In manycases it is obvious what surface features are responsible for the pointof interest. In this case, essentially all the points of interest areindependent in both position and value (though obviously tied to thesurface of the object). They are useful in identifying the individualturbine blade but useless in determining that the object is a turbineblade. Essentially all points of interest are dissimilar or “foreground”features.

In some embodiments, the image may be displayed in color to provide moreinformation. The dark line is actually in the image. In one example, afirst color may be used to display circles that are above a certainstrength or threshold value (“strong points of interest”), while asecond color may be used to show weaker points of interest, i.e., thosebelow the threshold value. In another example, additional colors, forexample, four or five colors, may be applied to points of interestcircles in a visual display to show a (quantized) range of strengthvalues. The “strength” of a point of interest is correlated to theability to find it under different conditions of angle and illumination.The use of visual displays including circles (or other shapes), colors,etc. may help human users to understand the computerized processes atwork.

Two different processes for detecting and measuring changes to an objectare described in more detail below. Which process to apply depends onthe category of objects to be considered. It should be kept in mind thatwhile two distinct categories are described herein, in practice, objectsgenerally fall within a range from one to the other. One example is coingrading, where both addition and subtraction may be equally important.While different approaches are described for the two categories, it isanticipated that for many objects a combination of approaches will berequired.

The system taught herein works when the change is negative orsubtractive—such as when parts of the surface abrade—as well as when thechange is additive. In either case, the match with the reference digitalfingerprint will degrade. In the former case, degradation is caused byremoval of the surface feature that was characterized as a point ofinterest in the reference fingerprint. In the latter case, that surfacefeature has been covered over by an additive process.

In one embodiment of this disclosure, features common to multipleexamples of a class of objects are identified and common points ofinterest are identified and characterized and then used for determiningchanges in members of that class of objects. At different times in theobject's life cycle its digital fingerprint is compared with the digitalfingerprint composed of the common points of interest and the change incorrespondence to those common points measured and used as a measurementof the change in the object. This is particularly useful when the objectbeing measured was not inducted (i.e. digitally fingerprinted) when new.

One advantage of the approach of using common points is the ability toproduce a degradation measuring system that does not require that aparticular object (of this kind) be inducted as a reference. Instead ofmeasuring degradation of a particular object from its earlier induction,the degradation in the object's match with a group digital fingerprintcomprising the common points is measured. An example of this type ofprocess follows.

Process for a Class of Objects with a Substantial Number of CommonPoints of Interest

FIG. 5 is a simplified flow diagram of a method to detect and measuredegradation in physical objects using digital fingerprints. A set of,preferably, new objects (although it is not strictly necessary that theobjects be new) that are all members of a class with a substantialnumber of common points is selected, block 502. The size of the set ischosen such that adding more members does not significantly increase thenumber of common points. Each member object of the set is inducted, andits digital fingerprint is extracted. Though the digital fingerprint maycontain many additional types of information, for the purposes of thisdisclosure, points of interest that have (up to invariances) the same,or very similar, location in multiple members of the class areconsidered. The exact number of times the point will occur in the classdepends on the number of objects in the class-specific data set and theconsistency and constitution of the background, but it has been foundthat, typically, if a point occurs in more than 14 members of such aclass, it is likely to occur in a great deal more (if the class is largeenough) and therefore is considered a common or “background” point orfeature.

Meaning of “up to invariances”. It was stated above that the commonpoints are in the same (or very similar) locations “up to invariances”.What this means depends on the type of invariance that the processing ofthe object achieves. Consider a dollar bill as an example. If an imageof a dollar bill is always captured in the same orientation and with thesame resolution, the common points of the dollar bill class will be invery nearly the same location in each image. If, however, the bill maybe imaged in any orientation, the location of the common points is “thesame” after correcting for the degree of rotation between member of theclass. If the image can be captured at different resolutions, the commonpoints are located in “the same location” up to a change in scale.Similar statements can be made for offset, affine, homography,projective, perspective, rubber sheet, and other variations. A point onone object is considered to be “in the same location” as a point onanother object of the same class if, when a matching of digitalfingerprints is performed, they are considered to be properly locatedfor a match. In this description, when it is stated that two points ondifferent class members are located in the same place, “up toinvariances” should always be understood to be implied.

Now that a set of common points of interest are defined, a “classdigital fingerprint” (or simply, “class fingerprint”) can be generated,so that the class fingerprint contains, preferably only, the commonpoints of interest, block 504. The class fingerprint can be used formatching an object to a class (that is, for determining that aparticular object is a member of a particular class whose classfingerprint the fingerprint of the object matches within a threshold).That is an important use, but it is not directly relevant to thissection of the description. The class fingerprint may have all thepoints found (including, say, 15 of each located very close to eachother), an average of the characteristics of each such point, or anyother way of combining them. Significantly, when the process iscompleted, the outcome will include a set of points, many of which arelikely to occur in any example of the object class.

For some applications, the selected set of objects should be objectsthat are as close as possible to a desired reference state. In manycases the preferred reference state will be “new”, but in the case of anobject inducted previously in its life cycle, the reference state mightbe the same object (that is, the digital fingerprint of the same object)acquired and stored at an initial or earlier induction. As mentioned,the objects are compared with each other and the points of interest thatare common to a significant number of them are preserved in a classfingerprint.

One benefit of having background features and using them to create aclass fingerprint is that the degree of change of a member of the classmay be determined without the individual object having been inductedwhen it was new. However well or poorly it matches the class fingerprint(compared to how well other items known to be worn out or in goodcondition matched and/or compared with its previous matches), measuresthe degree of wear.

Referring again to FIG. 5, next, a member of the class that was not partof the original set is selected, block 506. This selected member shouldpreferably be in a known condition, preferably in “new” condition,although any condition may be of use. A (first) digital fingerprint ofthe selected member is acquired, block 508. That digital fingerprint iscompared to the class fingerprint, block 510 and a measure of match isdetermined based on the comparison, block 512, and recorded, block 514.As the selected member experiences wear and tear, it is repeatedlymeasured against the class fingerprint. In FIG. 5, see block 516,creating a second digital fingerprint at a second time. Comparing thesecond digital fingerprint to the class digital fingerprint, block 518;determining a second measure of match, block 520; recording the secondmeasure of match, block 522; and comparing the first and second measuresof match to determine a deterioration of match metric, block 524.

The deterioration of the selected member's match to the classfingerprint is used as a measure or metric of its degree of change, seeblock 526. Determining that the object is worn out may either be done bytraining—that is, by seeing how much degradation other objects in theclass have undergone before a human decides they are worn out—or by apredetermined loss of matching points between the object and the classdigital fingerprint, or by some other method. When a sufficiently highlevel of degradation occurs in a tested object, the object is declaredto be worn out. These processes may be realized in software, forexample, in an analysis component 144 in the object change server 110 inFIG. 1.

Process for a Class of Objects without a Substantial Number of CommonPoints of Interest

In an embodiment an object is inducted when new (or other known state)and measurement of its change is based on how its own digitalfingerprint changes from the originally-inducted one. That digitalfingerprint may comprise only those points of interest that occur inthis object (and not in similar objects) or it may comprise all pointsof interest on the object, or, comprise different subsets of suchpoints.

It should be kept in mind that this approach may be used for any objectwhether or not it belongs to a class that has common points. It ispossible the use of common points of interest (in the former embodiment)as well as individual object points of interest in a given application.

When there are few or no “common features” or “background features”creating a class digital fingerprint is impossible, so the approach ofthe previous section is also impossible. When everything is foreground(or treated as though it were foreground), the object must be inductedwhen the object is in a known condition, preferably when it is new,although any condition may be acceptable. Change, therefore, is measuredas the change from the originally-inducted digital fingerprint. Aturbine blade is one example, but this approach works for almost anyobject, including those with substantial background points or features.

In this case an object is inducted when in a known condition, preferablyin “new” condition for measuring degradation, or, as another example,“prior to anodizing” if anodizing will be taking place, and so on.Later, when the object has been worn or anodized or otherwise altered,the object is inducted again to measure the difference in match pointsfrom when it was initially inducted.

FIG. 4 is a simplified flow diagram illustrating a method to detect andmeasure degradation in physical objects using digital fingerprints,without a class digital fingerprint. At block 402, the process calls forinducting a physical object in a known initial condition into a digitalfingerprinting system. Inducting the physical object includes generatingand storing a first digital fingerprint of the object at a first time,block 406. This may be called a reference digital fingerprint. Nextcomes re-inducting the object at a second or next time after the firsttime, to form a second or next digital fingerprint of the object, block408. The second or next digital fingerprint is stored in the datastore,block 410. In this description and the claims, phrases like “asecond/next digital fingerprint” and “a first/next comparison” etc. areused to indicate a repeating process, where the “next digitalfingerprint” for example, would correspond to a second, third, fourth,etc. digital fingerprint as a repeating process or “loop” is executed.Such a loop is indicated in FIG. 4 from the decision 418 back to block408.

At block 412, the process continues by comparing the second/next digitalfingerprint of the object to a first/next preceding digital fingerprintto form a first/next comparison. Block 414 calls for determining afirst/next measure of change of the object between the first/next timeand the second/next subsequent time based on the first/next comparison.Block 416 calls for recording the first/next measure of change toaccumulate measures of change of the object over time, namely, from thefirst time to the last (most recent) next time.

A decision 418 determines whether the manufacturing process hascompleted. If so, the process may terminate, block 422. If not, theillustrated method may provide feedback to the manufacturing process,block 420. That is, it may provide information based on changes to theobject during the manufacturing process. for example, if a platingprocess is specified to continue until a certain measure of change isaccomplished, the feedback may indicate that the measure of change hasoccurred and thus the plating process is completed. Assuming the processis not completed, the method loops back to block 408 to again re-inductthe object.

Training

Determining how much measured change is sufficient to, say, remove abill or turbine blade from service or show that the anodizing processhas been successful can be done in various ways, any of which are inview in this disclosure. Two approaches are worth highlighting. First, adegree of change of the test digital fingerprint from the reference onemay be set. To use the wear-and-tear example of a turbine blade, when,for example, half the points originally seen are no longer there tomatch the reference, that level of change may be defined as a prompt toreplace the blade. This makes sense because half the points missing isindicative of half the surface having been substantially altered. Incategory one (the category with a large number of background points) itis indicative that half the points that make the object a dollar bill(or whatever) are missing so it may be time to recycle the bill. Notethat the use of “half” here is purely arbitrary. The number of pointsmay be set at any level and by any manner, including manually, as aresult of experience, as a result of a training process, or by any othermethod.

The training process approach involves training the system to determinethe action to take based on a given degree of change. Again, the wearand tear surface erosion example will be used, but the example is notmeant to be limiting. One way to do the training is to induct a set ofobjects and follow them through their life cycle, measuring them as timepasses until they are ready to be replaced. The degradation score attheir points of replacement (or the aggregate or average of such scores)may then be used to define a norm or a standard. A more sophisticatedtraining system may be envisioned where a neural net or other machinelearning system is trained with objects at various parts of their lifecycles and, after the training cycle is complete, the match of theobject to the class digital fingerprint (first category) or to itsinitial induction (second category) is put into the learning system andthe system indicates, for example, how much life is left in the objector that it is time to replace the object. A similar training system maybe envisioned for use in additive processes as well.

Non-Limiting Sample Use Cases

In addition to the currency bill and turbine blade examples exploredabove, many other embodiments are possible. The teachings of thisdisclosure apply wherever measurement of change in the surface featuresof a physical object is useful.

Plating. In addition to anodizing, mentioned above, other methods ofobscuring the surface are in view. The obscuring may be unintentional orintentional. Plating is one example. Depending on how thick the platingis supposed to be, the object may be inducted once or many times in theplating process.

In an embodiment, an object's surface may need to be covered with newmaterial that is too thick to see through. There are two uses fordigital fingerprinting here. First, the teachings of this disclosure canbe used to determine the progress and effectiveness of such additions.In this case, each time the previous surface “disappears” to somepredetermined amount, the object is re-inducted, and a new referencepreserved. When that surface in turn partially “disappears”, furtherinduction is performed and so on until the addition process is complete.The second use is to re-induct the part when the new surface is finishedso that, going forward, the part's identity can be established. As such,in this embodiment the teachings of this disclosure are used to bothensure the additive process is working correctly and, when that processis finished, to ensure that the system can continue to identify the partas it moves through further manufacturing stages.

Item aging. Aging here is a stand-in term for conditions where theobject changes not because it is added to or abraded but simply as aresult of the passage of time, its interaction with the environment, itsregrowth/regeneration (such as in the case of skin or living material),and so on. In this case, surface points may be added or subtracted bythe aging process and the degree of change measured not just in terms ofpoints of interest lost but also points of interest that first appearbetween inductions.

Coin (and other object) grading. Many objects are graded to determinehow closely they conform to a standard object. Coins are examples ofthis. In general, the more the coin is worn (in the case of circulatedcoins) or scarred by, say, other coins in a bag (uncirculated coins),the less well they will conform.

Coins are graded somewhat differently depending on their type. Proofcoins are (generally) double stamped and stamped with mirrored dies.They are not meant for general circulation. Proof coins are graded fromPR-60 to PR-70 (impaired or rubbed proof coins may have a lower grade).Uncirculated regular coins are coins that have never been in generalcirculation. Uncirculated coin grades run from MS-60 to MS-70, whichcorrespond in quality and amount of degradation to the PR categories forproof coins.

Coins that have been in general circulation are graded in a range from 1to 59. 50 and up are called “about uncirculated” (AU). 40-49 are“extremely fine” (XF). 20-39 are “very fine” (VF). Below that are “fine”(F), “very good” (VG), “good” (G), “about good” (AG), “fair” (FR),“poor” (P), and “ungradable”. Examples are given below.

Coins fit into the first category of objects with similar backgroundfeatures, described above as coins have a large quantity of shared“background” points. It is clear from the images that degradation frominitial conditions depends on both addition (patina, scratch marks) andsubtraction (wear marks). The taught system may be used to carry outcoin grading by having coins graded by an expert and the digitalfingerprint of the coins extracted. A learning system is then programmedbased on the grade and the measured change from an ideal example of thatcoin type. When a coin of that type is to be graded, its digitalfingerprint is extracted and additions and subtractions from thestandard measured, the results fed through the learning system, and thegrade produced.

Digital Fingerprinting

“Digital fingerprinting” refers to the creation and use of digitalrecords (digital fingerprints) derived from properties of a physicalobject, which digital records are typically stored in a database.Digital fingerprints maybe used to reliably and unambiguously identifyor authenticate corresponding physical objects, track them throughsupply chains, record their provenance and changes over time, and formany other uses and applications.

Digital fingerprints store information, preferably in the form ofnumbers or “feature vectors,” that describes features that appear atparticular locations, called points of interest, of a two-dimensional(2-D) or three-dimensional (3-D) object. In the case of a 2-D object,the points of interest are preferably on a surface of the correspondingobject; in the 3-D case, the points of interest may be on the surface orin the interior of the object. In some applications, an object “featuretemplate” may be used to define locations or regions of interest for aclass of objects. The digital fingerprints may be derived or generatedfrom digital data of the object which may be, for example, image data.

While the data from which digital fingerprints are derived is oftenimages, a digital fingerprint may contain digital representations of anydata derived from or associated with the object. For example, digitalfingerprint data may be derived from an audio file. That audio file inturn may be associated or linked in a database to an object. Thus, ingeneral, a digital fingerprint may be derived from a first objectdirectly, or it may be derived from a different object (or file) linkedto the first object, or a combination of the two (or more) sources. Inthe audio example, the audio file may be a recording of a personspeaking a particular phrase. The digital fingerprint of the audiorecording may be stored as part of a digital fingerprint of the personspeaking. The digital fingerprint (of the person) may be used as part ofa system and method to later identify or authenticate that person, basedon their speaking the same phrase, in combination with other sources.

Returning to the 2-D and 3-D object examples mentioned above, featureextraction or feature detection may be used to characterize points ofinterest. In an embodiment, this may be done in various ways. Twoexamples include Scale-Invariant Feature Transform (or SIFT) and SpeededUp Robust features (or SURF). Both are described in the literature. Forexample: “Feature detection and matching are used in image registration,object tracking, object retrieval etc. There are number of approachesused to detect and matching of features as SIFT (Scale Invariant FeatureTransform), SURF (Speeded up Robust Feature), FAST, ORB etc. SIFT andSURF are most useful approaches to detect and matching of featuresbecause of it is invariant to scale, rotate, translation, illumination,and blur.” MISTRY, Darshana et al., Comparison of Feature Detection andMatching Approaches: SIFT and SURF, GRD Journals—Global Research andDevelopment Journal for Engineering |Volume 2|Issue 4|March 2017.

In an embodiment, features may be used to represent information derivedfrom a digital image in a machine-readable and useful way. Features maybe point, line, edges, and blob of an image etc. There are areas asimage registration, object tracking, and object retrieval etc. thatrequire a system or processor to detect and match correct features.Therefore, it may be desirable to find features in ways that areinvariant to rotation, scale, translation, illumination, noisy and blurimages. The search of interest points from one object image tocorresponding images can be very challenging work. The search maypreferably be done such that same physical interest points can be foundin different views. Once located, points of interest and theirrespective characteristics may be aggregated to form the digitalfingerprint (generally including 2-D or 3-D location parameters).

In an embodiment, features may be matched, for example, based on findinga minimum threshold distance. Distances can be found using Euclideandistance, Manhattan distance etc. If distances of two points are lessthan a prescribed minimum threshold distance, those key points may beknown as matching pairs. Matching a digital fingerprint may compriseassessing a number of matching pairs, their locations or distance andother characteristics. Many points may be assessed to calculate alikelihood of a match, since, generally, a perfect match will not befound. In some applications an “feature template” may be used to definelocations or regions of interest for a class of objects.

Scanning

In this application, the term “scan” is used in the broadest sense,referring to any and all means for capturing an image or set of images,which may be in digital form or transformed into digital form. Imagesmay, for example, be two dimensional, three dimensional, or in the formof a video. Thus a “scan” may refer to an image (or digital data thatdefines an image) captured by a scanner, a camera, a specially adaptedsensor or sensor array (such as a CCD array), a microscope, a smartphonecamera, a video camera, an x-ray machine, a sonar, an ultrasoundmachine, a microphone (or other instruments for converting sound wavesinto electrical energy variations), etc. Broadly, any device that cansense and capture either electromagnetic radiation or mechanical wavethat has traveled through an object or reflected off an object or anyother means to capture surface or internal structure of an object is acandidate to create a “scan” of an object. Various means to extract“fingerprints” or features from an object may be used; for example,through sound, physical structure, chemical composition, or many others.The remainder of this application will use terms like “image” but whendoing so, the broader uses of this technology should be implied. Inother words, alternative means to extract “fingerprints” or featuresfrom an object should be considered equivalents within the scope of thisdisclosure. Similarly, terms such as “scanner” and “scanning equipment”herein may be used in a broad sense to refer to any equipment capable ofcarrying out “scans” as defined above, or to equipment that carries out“scans” as defined above as part of their function.

More information about digital fingerprinting can be found in variouspatents and publications assigned to Alitheon, Inc. including, forexample, the following: DIGITAL FINGERPRINTING, U.S. Pat. No.8,6109,762; OBJECT IDENTIFICATION AND INVENTORY MANAGEMENT, U.S. Pat.No. 9,152,862; DIGITAL FINGERPRINTING OBJECT AUTHENTICATION ANDANTI-COUNTERFEITING SYSTEM, U.S. Pat. No. 9,443,298; PERSONAL HISTORY INTRACK AND TRACE SYSTEM, U.S. Pat. No. 10,037,537; PRESERVINGAUTENTICATION UNDER ITEM CHANGE, U.S. Pat. App. Pub. No. 2017-0243230A1. These references are incorporated herein by this reference.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the disclosure to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the disclosure and its practical applications, to therebyenable others skilled in the art to best utilize the disclosure andvarious embodiments with various modifications as are suited to theparticular use contemplated.

The system and method disclosed herein may be implemented via one ormore components, systems, servers, appliances, other subcomponents, ordistributed between such elements. When implemented as a system, suchsystems may include an/or involve, inter alia, components such assoftware modules, general-purpose CPU, RAM, etc. found ingeneral-purpose computers. In implementations where the innovationsreside on a server, such a server may include or involve components suchas CPU, RAM, etc., such as those found in general-purpose computers.

Additionally, the system and method herein may be achieved viaimplementations with disparate or entirely different software, hardwareand/or firmware components, beyond that set forth above. With regard tosuch other components (e.g., software, processing components, etc.)and/or computer-readable media associated with or embodying the presentdisclosure, for example, aspects of the innovations herein may beimplemented consistent with numerous general purpose or special purposecomputing systems or configurations. Various exemplary computingsystems, environments, and/or configurations that may be suitable foruse with the innovations herein may include, but are not limited to:software or other components within or embodied on personal computers,servers or server computing devices such as routing/connectivitycomponents, hand-held or laptop devices, multiprocessor systems,microprocessor-based systems, set top boxes, consumer electronicdevices, network PCs, other existing computer platforms, distributedcomputing environments that include one or more of the above systems ordevices, etc.

In some instances, aspects of the system and method may be achieved viaor performed by logic and/or logic instructions including programmodules, executed in association with such components or circuitry, forexample. In general, program modules may include routines, programs,objects, components, data structures, etc. that perform particular tasksor implement particular instructions herein. The disclosures may also bepracticed in the context of distributed software, computer, or circuitsettings where circuitry is connected via communication buses, circuitryor links. In distributed settings, control/instructions may occur fromboth local and remote computer storage media including memory storagedevices.

The software, circuitry and components herein may also include and/orutilize one or more type of computer readable media. Computer readablemedia can be any available media that is resident on, associable with,or can be accessed by such circuits and/or computing components. By wayof example, and not limitation, computer readable media may comprisecomputer storage media and communication media. Computer storage mediaincludes volatile and nonvolatile, removable and non-removable mediaimplemented in any method or technology for storage of information suchas computer readable instructions, data structures, program modules orother data. Computer storage media includes, but is not limited to, RAM,ROM, EEPROM, flash memory or other memory technology, CD-ROM, digitalversatile disks (DVD) or other optical storage, magnetic tape, magneticdisk storage or other magnetic storage devices, or any other mediumwhich can be used to store the desired information and can accessed bycomputing component. Communication media may comprise computer readableinstructions, data structures, program modules and/or other components.Further, communication media may include wired media such as a wirednetwork or direct-wired connection, however no media of any such typeherein includes transitory media. Combinations of the any of the aboveare also included within the scope of computer readable media.

In the present description, the terms component, module, device, etc.may refer to any type of logical or functional software elements,circuits, blocks and/or processes that may be implemented in a varietyof ways. For example, the functions of various circuits and/or blockscan be combined with one another into any other number of modules. Eachmodule may even be implemented as a software program stored on atangible memory (e.g., random access memory, read only memory, CD-ROMmemory, hard disk drive, etc.) to be read by a central processing unitto implement the functions of the innovations herein. Or, the modulescan comprise programming instructions transmitted to a general-purposecomputer or to processing/graphics hardware via a transmission carrierwave. Also, the modules can be implemented as hardware logic circuitryimplementing the functions encompassed by the innovations herein.Finally, the modules can be implemented using special purposeinstructions (SIMD instructions), field programmable logic arrays or anymix thereof which provides the desired level performance and cost.

As disclosed herein, features consistent with the disclosure may beimplemented via computer-hardware, software and/or firmware. Forexample, the systems and methods disclosed herein may be embodied invarious forms including, for example, a data processor, such as acomputer that also includes a database, digital electronic circuitry,firmware, software, or in combinations of them. Further, while some ofthe disclosed implementations describe specific hardware components,systems and methods consistent with the innovations herein may beimplemented with any combination of hardware, software and/or firmware.Moreover, the above-noted features and other aspects and principles ofthe innovations herein may be implemented in various environments. Suchenvironments and related applications may be specially constructed forperforming the various routines, processes and/or operations accordingto the present disclosure or they may include a general-purpose computeror computing platform selectively activated or reconfigured by code toprovide the necessary functionality. The processes disclosed herein arenot inherently related to any particular computer, network,architecture, environment, or other apparatus, and may be implemented bya suitable combination of hardware, software, and/or firmware. Forexample, various general-purpose machines may be used with programswritten in accordance with teachings of the present disclosure, or itmay be more convenient to construct a specialized apparatus or system toperform the required methods and techniques.

Aspects of the method and system described herein, such as the logic,may also be implemented as functionality programmed into any of avariety of circuitry, including programmable logic devices (“PLDs”),such as field programmable gate arrays (“FPGAs”), programmable arraylogic (“PAL”) devices, electrically programmable logic and memorydevices and standard cell-based devices, as well as application specificintegrated circuits. Some other possibilities for implementing aspectsinclude memory devices, microcontrollers with memory (such as EEPROM),embedded microprocessors, firmware, software, etc. Furthermore, aspectsmay be embodied in microprocessors having software-based circuitemulation, discrete logic (sequential and combinatorial), customdevices, fuzzy (neural) logic, quantum devices, and hybrids of any ofthe above device types. The underlying device technologies may beprovided in a variety of component types, e.g., metal-oxidesemiconductor field-effect transistor (“MOSFET”) technologies likecomplementary metal-oxide semiconductor (“CMOS”), bipolar technologieslike emitter-coupled logic (“ECL”), polymer technologies (e.g.,silicon-conjugated polymer and metal-conjugated polymer-metalstructures), mixed analog and digital, and so on.

It should also be noted that the various logic and/or functionsdisclosed herein may be enabled using any number of combinations ofhardware, firmware, and/or as data and/or instructions embodied invarious machine-readable or computer-readable media, in terms of theirbehavioral, register transfer, logic component, and/or othercharacteristics. Computer-readable media in which such formatted dataand/or instructions may be embodied include, but are not limited to,non-volatile storage media in various forms (e.g., optical, magnetic orsemiconductor storage media) though again does not include transitorymedia. Unless the context clearly requires otherwise, throughout thedescription, the words “comprise,” “comprising,” and the like are to beconstrued in an inclusive sense as opposed to an exclusive or exhaustivesense; that is to say, in a sense of “including, but not limited to.”Words using the singular or plural number also include the plural orsingular number respectively. Additionally, the words “herein,”“hereunder,” “above,” “below,” and words of similar import refer to thisapplication as a whole and not to any particular portions of thisapplication. When the word “or” is used in reference to a list of two ormore items, that word covers all of the following interpretations of theword: any of the items in the list, all of the items in the list and anycombination of the items in the list.

Although certain presently preferred implementations of the presentdisclosure have been specifically described herein, it will be apparentto those skilled in the art to which the present disclosure pertainsthat variations and modifications of the various implementations shownand described herein may be made without departing from the spirit andscope of the present disclosure. Accordingly, it is intended that thepresent disclosure be limited only to the extent required by theapplicable rules of law.

While the foregoing has been with reference to a particular embodimentof the disclosure, it will be appreciated by those skilled in the artthat changes in this embodiment may be made without departing from theprinciples and spirit of the disclosure, the scope of which is definedby the appended claims.

1. A method comprising: selecting a set of physical objects that are allmembers of a class of objects that have at least a predetermined numberof points of interest in common, wherein the points of interest of eachobject are identified by a digital fingerprinting process that extractssurface or near-surface features, based on digital image data of thecorresponding object; generating a class digital fingerprint for theclass of objects, the class digital fingerprint containing the commonpoints of interest; selecting an object that is a member of the class ofobjects but is not a member of the selected set; at a first time,creating a first digital fingerprint of the selected object; comparingthe first digital fingerprint of the selected object to the classdigital fingerprint to form a first comparison; determining a firstmeasure of match based on the first comparison, wherein the firstmeasure of match is based on counting a number of points of interestfound in the first digital fingerprint that match corresponding pointsof interest in the class digital fingerprint; recording the firstmeasure of match; at a second time, creating a second digitalfingerprint of the selected object; comparing the second digitalfingerprint to the class digital fingerprint to form a secondcomparison; determining a second measure of match based on the secondcomparison, wherein the second measure of match is based on counting anumber of points of interest found in the second digital fingerprintthat match corresponding points of interest in the class digitalfingerprint; comparing the first and second measures of match todetermine a deterioration of match metric; and calculating and storing ameasure of change of the selected object during a time period betweenthe first time and the second time based on the deterioration of matchmetric.
 2. The method of claim 1 wherein creating the first and seconddigital fingerprints includes extracting a least one location ofinterest in the corresponding digital image data, the location ofinterest based on the image data and not selected at random, generatingfeature vectors based on the at least one location of interest, andusing the feature vectors to form the digital fingerprint.
 3. The methodof claim 2 further comprising: comparing the measure of change of changeof the object to a predetermined threshold change value; and executing apredetermined action in a case that the measure of change exceeds thepredetermined threshold change value.
 4. The method of claim 2 whereinthe comparing the first/second digital fingerprint to the class digitalfingerprint are based on counting a number of points of interest foundin the first/second digital fingerprint that match corresponding pointsof interest in the class digital fingerprint.
 5. The method of claim 4wherein matching points of interest are determined by: calculating afeature vector distance between each point of interest and acorresponding point of interest in the class digital fingerprint; anddesignating as matching those points of interest where the featurevector distance is less than a predetermined threshold value.
 6. Themethod of claim 1 including: storing the digital fingerprints of thefirst physical object in a datastore; adding to each of the storeddigital fingerprints corresponding metadata including a date-time stampand geographic location data where image data was captured to form thedigital fingerprint; based on the stored digital fingerprints and themetadata, building a conceptual map of geographic locations wherechanges to the object have occurred; and generating output to a userinterface based on the conceptual map.
 7. The method of claim 1including: comparing the measure of change to a predetermined thresholdvalue; and selecting and executing an action based on the comparison. 8.The method of claim 1 including: identifying among the class memberscertain points of interest that occur (up to invariances) at the same ora very similar location in the corresponding digital fingerprints; anddesignating as points of interest in the class digital fingerprint onlythose certain points of interest that occur in more than about 14 of theclass members.
 9. The method of claim 1 wherein the class of physicalobjects is a coin, and the set of physical objects comprises a pluralityof individual coins all having a common issuer face value, year, anddesign.
 10. A method comprising: inducting a physical object in a knowninitial condition into a digital fingerprinting system, whereininducting the physical object includes generating and storing a firstdigital fingerprint of the object as a reference digital fingerprint ata first time; at a second time after the first time, generating andstoring a second digital fingerprint of the object; comparing the seconddigital fingerprint to the reference digital fingerprint to form a firstcomparison; determining a metric of change of the object between thefirst time and the second time based on the first comparison; comparingthe metric of change to a predetermined threshold value; and taking apredetermined action responsive to the comparison to the thresholdvalue.
 11. The method of claim 10 wherein the physical object isundergoing a manufacturing process, and further comprising transmittinga feedback message to the manufacturing process based on the metric ofchange to the physical object.
 12. The method of claim 11 including:determining a conclusion of the processing of the physical object toform a finished physical object; and inducting the finished physicalobject into the database system.
 13. The method of claim 11 wherein theprocessing is an additive process, whereby some points of interest ofthe object are covered and therefore disappear from a given inductionrelative to an earlier induction, and the method further includes:assessing progress of the additive process based on at least one of thestored measures of change to the object; and adjusting the processing ofthe physical object based on the assessment.
 14. The method of claim 13wherein the additive process includes plating material onto the physicalobject.
 15. The method of claim 10 and further comprising: inducting aset of objects; follow the set of objects through their life cycle,capturing the metrics of change as time passes until at least some ofthe set of objects are designated to be replaced; identifying thecaptured metrics of change of the objects designated to be replaced; anddetermining the threshold value based on the identified metrics.
 16. Themethod of claim 15 including: determining an average value of thecaptured metrics of change of the objects designated to be replaced; andutilizing the average value as the threshold value.
 17. A systemcomprising: means to receive or generate digital fingerprints ofphysical objects, wherein each digital fingerprint is based on at leastone digital image of the corresponding physical object and includes databased on the digital image(s) that describe plural points of interestfound on or near a surface of the corresponding physical object; anobject change server coupled to the input means and including aprocessor; a datastore operatively coupled to the object change server;and a user interface operatively coupled to the object change server;wherein the processor has access to a memory storing machine-readableinstructions executable on the processor, the instructions arranged tocause the processor to: store a first digital fingerprint received fromthe input means into the datastore; query the datastore to access asecond digital fingerprint that matches the first digital fingerprint;compare the first digital fingerprint to the second digital fingerprintto form a comparison; determine a measure of change to the physicalobject based on the comparison; generate an output to the user interfacebased on the measure of change.
 18. The system of claim 17 wherein: thefirst digital fingerprint is extracted from a first physical object is amember of a class of objects having common points of interest; thesecond digital fingerprint is a class fingerprint based on a set ofobjects that are all members of the class; whereby the output indicatesa change of the first physical object relative to the class digitalfingerprint.
 19. The system of claim 17 wherein the instructions arefurther arranged to cause the processor to: receive or generate at leastone additional digital fingerprint of the first physical object, whereinthe additional digital fingerprint is based on image data of the firstphysical object that is acquired by a scanner at second time that issubsequent to a first time when the first digital fingerprint wasacquired; compare the additional digital fingerprint to the seconddigital fingerprint to form a second comparison; determine a secondmeasure of change to the physical object based on the second comparison;and generate an output to the user interface based on the first andsecond measures of change.
 20. The system of claim 19 wherein the outputto the user interface includes an indication of wear of the physicalobject between the first time and the second time.